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I ,   HISTORICAL  INTRODUCTION 

In  the  last  two  decades  with  the  school-age  population  expansi 
sorely  testing  the  American  egalitarian  ideal  of  universal  education, 
increasing  attention  has  "been  devoted  to  the  techniques  of  instruction. 
One  technology  resurrected  as  a  result  of  this  al  ' <  n  Lon  has  been  the  idea 
of  the  teaching  machine.   This  device,  it  was  hoped,  would  relieve  th< 
classroom  teacher  of  the  teaching  drudgery  and  allow  the  teacher  to  spend 
more  time  helping  pupils  on  an  Individual  basis. 

The  first  serious  attempt  to  use  a  teaching  machine  was  made  by 
S.L.  Pressey,   During  the  1920' s  Pressey  published  numerous  papers  about  a 
machine  he  had  developed.   This  machine  was  about  the  size  of  a  typewriter, 
was  capable  of  asking  multiple  choice  questions  and  could  grade  the  answers. 
[60,  6l,  62]  .   Prior  to  this  time  many  other  teaching  devices  had  been 
patented  (over  600  between  1809  and  1936)  but  no  investigator  had  tried  to 
interest  the  public  in  these  devices.   Although  Pressey  spent  much  of  the 
next  30  years  trying  to  gain  public  acceptance  of  his  ideas,  it  remained  for 
B,  F.  Skinner  to  actually  get  educators  stirred  up  over  the  teaching 
machine  concept . 

B.  F.  Skinner,  who  developed  the  "Skinner  Box"  and  used  it  in  his 
investigation  of  animal  learning,  felt  that  the  same  techniques  which  he 
employed   in  the  laboratory  on  animals  could  also  be  used  in  the  schools  on 
people  [79]  •   Using  these  techniques  he  developed  a  teaching  machine  which 
would  display  the  material  to  be  learned  in  small  segments  called  "frames". 
These  frames  were  ordered  in  a  logical  sequence  and  each  contained  some 
statement  with  blanks  or  a  question  which  required  the  student  to  construct 
a  written  response.   As  soon  as  the  student  finished  writing  the  answer  to 
the  frame  he  was  given  the  correct  answer  for  comparison  purposes,   The 
frames  were  written  so  that  using  knowledge  from  previous  frames  and  cues 
from  the  present  one,  the  student  was  almost  certain  to  get  the  right  answer. 
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This  correct  response  was  intended  to  reinforce  the  learning  of  that  particular 
frame 

An  alternative  to  Skinner's  method  came  from  N,  A.  Crowder  who  felt 
tha+  Skinner's  method  was  too  inflexible  in  its  handling  of  the  student  .[19]  . 
He  de^elcped  some  programs  which  branched  depending  on  the  answer  the  student 
made  to  the  last  question,   A  wrong  answer  to  a  question  caused  a  branch  to 
a  special  sequence  of  frames  which  dealt  with  the  specific  mistake  made  by 
the  student.   A  correct  answer,  on  the  other  hand,  led  to  the  next  instructional 
item.   This  method  of  programming  is  known  as  branching  or  intrinsic  programming 
as  opposed  to  Skinner's  linear  programming  in  which  there  is  only  one  path 
through  the  program-   Crowder  hoped  that  by  using  branching,  the  advanced 
students  who  learned  more  quickly  would  be  able  to  skip  the  more  rudimentary 
material  incorporated  for  the  less  well-prepared  or  slower-learning  students. 

The  branching  program  represenx-s  a  completely  different  concept  of 
the  teaching  (learning)  process  from  Skinner's  linear  program:   Skinner 
viewed  the  teaching  machine  as  a  means  of  conditioning  a  student  to  emit  the 
appropriate  response  (answer)  when  faced  with  a  given  stimulus  (question). 
Crowder,  on  the  other  hand,  felt  that  the  educational  process  was  basically 
a  feedback  situation  in  which  a  teacher  presents  new  material  to  the  student 
and  then  determines  if  he  has  learned  it,   If  not  the  teacher  reviews  the 
material  or  presents  a  corrective  sequence  before  going  on. 

A  third  programming  method,  called  adaptive  programming,  usually 
involves  the  use  of  a  computer  or  other  complex  electronic  device,   In  this 
method  the  machine  takes  into  account  various  individual  characteristics  such 
as  pers     ry  (aggressiveness,  shyness,  etc.  )>   IQ  and  also  student  behavior 
on  previous  questions  to  decide  the  type  of"  sequences  to  present  to  the 
student.   Gordan  Pask  of  Great  Britain  helped  design  one  such  machine,  SAKI. 
Another,  SOCRATES,  has  been  programmed  on  a  computer  at  the  Illinois  Training 
Research  Laboratory  [36]. 
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Adaptive  programming  is  sometimes  also  i  aJ  Led  extrinsic  since 
uses  information  external  to  the  program,  (IQ,  personality  scores,  apti- 
tudes, etc.)  in  making  its  sequencing  decisions;  this  is  in  opposition  ' 
branching  or   intrinsic  programming,  which  only  uses  the  response  to  the 
last  question  [20,  25]. 

When  Skinner  began  advocating  his  machine  the  economic  situation 
and  the  teacher  shortage  caused  immediate  interest  to  develop.   Cons     ible 
>arch  was  started  to  determine  the  best  types  of  teaching  programs  and  to 
compare  the  amount  of  learning  produced  by  this  technique  with  that  found  in 
the  ordinary  classroom  situation.   At  the  same  time  the  educational  press 
became  interested  and  many  articles  were  written  both  pro  and  con  [  1  k ,  32, 
33,  h6,    55,  56,  92,  93],   Partly  as  a  result  of  this  publicity  there  was  a 
spurt  in  the  actual  production  of  machines  despite  the  fact  that  the  theory 
of  these  had  not  been  completely  worked  out ,   W.  L.  Ross  et-al.  1 69]  list 
some  UO  teaching  machines  in  their  buyer's  guide  published  in  1962-   Finn 
and  Perrin   [31]  ,  also  in  1962,  list  almost  100.   These  machines  ranged 
in  price  from  $1  to  $1600  with  one  machine  priced  at  $5000.   The  machines 
consisted  of  everything  from  manual  crank  devices  which  contained  the  program 
on  a  strip  of  paper,  to  complex  devices  which  showed  slides  and  played 
recorded  instruction  via  tape  recorders. 

It  scon  became  apparent,  however,  that  without  good  programs  these 
machines  were  not  appropriate  for  large-scale  school  use.   Although  many 
programs  were  produced,  the  overall  quality  was  not  always  good  and  the 
measurement  of  program  quality  became  an  important,  topic  [U,  15,  ^0,  kQ ,    58, 
183].   This,  coupled  with  the  high  cost  of  the  machines  in  many  instances 
and  the  fact  that  changes  in  programming  theory  or  technique  quickly  made 
machines  obsolete,  took  the  bloom  off  the  rosy  expectations  that  the 
manufactures  had  had  for  the  middle  and  late  1960's,   Although  school 
systems  have  indeed  become  interested  in  programmed  learning,  their  plans 
for  large  scale  integration  of  teaching  machines  still  appear  to  be  a  few 
years  in  the  future, [28], 
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During  the  late  1950  's  the  computer  industry  "became  interested  in 
programmed  instruction.   A  variety  of  "computer  aided  instruction"  (CAI) 
projects  were  begun  to  utilize  the  high-speed  computer  as  an  instruction 
control  device 

As  early  as  1958  the  IBM  Research  Laboratories  began  a  project 
tc  teach  binary  arithmetic  using  an  IBM  650  computer  and  typewriter  console 
[65,    17*0  •   The  computer  was  programmed  to  give  problems  to  the  student  and 
then  grade  his  answer,   On  the  basis  of  the  correctness  of  the  answer  the 
somputer  would  then  generate  another  problem.   Later  the  IBM  group  built  an 
arithmetic  tutor  for  6th  grade  students.   This  project  in  turn  led  to 
further  experiments  in  teaching  a  broader  range  of  courses  such  as  steno- 
typmg,  statistics  and  German  reading  [9,  17^,  196]-   Eventually  in  19&5 
IBM  announced  the  development  of  a  compiler-like  language,  called  COURSEWRITER 
[36] , which  allowed  a  teacher  to  program  a  computer  to  teach  his  own  subject. 

In  early  i960  the  Systems  Development  Corporation  began  work  in 
automated  teaching  using  a  Bendex  G-15  computer  [il6,  117,  119»  123]  >   They 
experimented  with  various  parameters  and  this  eventually  led  to  their  develop- 
ment of  CLASS  (Computer-Based  Laboratory  for  Automated  School  Systems),  an 
educational  facility  designed  to  explore  the  potential  of  a  fully  integrated 
classroonic   This  scheme  combined  automated  instruction  with  other  instruc- 
tional methods  such  as  films,  TV,  lectures  and  textbooks  [9,  117 »  123]. 
CLASS  is  designed  to  accommodate  as  many  as  20  students  in  a  variety  of 
modes,   Its  main  purpose  is  to  test  the  new  techniques  in  an  integrated 
manner  similar  to  the  way  they  might  eventually  be  used  in  a  regular 
classroom- 

Also  in  i960  the  PLATO  project  (Programmed  Logic  for  Automatic 
Teaching  Organization)  at  the  University  of  Illinois  was  begun,,  [98s  99, 
100,  10 1 ,  103] ■   Under  the  direction  of  D.  L.  Bitzer  and  P„  G.  Braunfeid, 
its  original  purpose  was  to  investigate  student  behavior  when  in  contact 
with  a  teaching  machine.   It  began  with  two  terminals  attached  to  the 
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Illiac  I  computer,   Today  after  extensive  modifications  the  system  uses  a 
CDC  l60U  and  controls  20  consoles   Each  console  has  a  TV  device  which  can 
display  slides  or  computer  generated  output  -  all  under  control  of  the 
computer   The  student  is  equipped  with  a  television  monitor,  a  typewriter 
key  set  and  certain  other  control  "buttons  with  which  he  can  "talk"  to  the 
computer.   A  wide  variety  of  courses  have  been  programmed  and  taught  to 
students . 

A  good  example  of  a  computer  oriented  adaptive  system  is  the 
SOCRATES  II  system  (System  for  Organizing  Content  to  Review  and  Teach 
Educational  Subjects)  [23,  36,  193].   Built  by  the  Illinois  Training 
Research  Laboratory,  the  system  controlled  several  teaching  machine 
stations.   The  system  is  initially  provided  data  on  each  student's  IQ, 
aptitude  scores,  personality  traits  and  other  relevant  information-   Using 
this  it  determines  the  strategy  of  instruction  to  give  to  each  student 

There  are  various  other  computer  systems  including  COBIS  (Computer 
Based  Instructional  System)  at  the  Decision  Sciences  Laboratory  at  Hanscom 
Field,  the  TRW  MENTOR  which  uses  films  and  audio  magnetic  tape  recorders 
[9>  13],  and  the  Stanford  I,  built  at  Stanford  University  to  control  audio 
and  video  stimuli  by  computer  [200].   One  other  example  of  an  automated 
teaching  system,  which  shows  another  direction  of  research,  is  an  earlier 
project  at  the  Decision  Sciences  Laboratory  [159] .   This  project  developed 
at  about  the  same  time  as  the  beginning  of  the  IBM  project  but  it  was 
oriented  on  a  more  pragmatic  basis .   In  this  case  the  basic,  idea  was  to 
use  an  existing  large-scale  computer  system  to  help  teach  its  potential 
users  how  to  employ  it.   A  combination  of  programmed  text  and  computer 
teaching  was  used  without  human  intervention. 

In  the  next  section  we  will  go  back  to  the  early  cont  oversies 
between  linear  and  branching  programming  advocates.   Although  this  contro- 
versy is  currently  dying  out  because  of  the  willingness  of  present  day 
programmers  to  use  whatever  is  best  for  a  given  application  [5^5  56] ,  it 
is  still  important  historically  since  it  led  to  a  great  deal  of  research 
effort  on  the  part  of  the  various  advocates.   Although  much  of  this  research 
gave  conflicting  results  it  represents  the  inaugeration  of  a  large-scale 
effort  to  discover  the  effective  variables  in  programmed  learning, 
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II.   PAST  CONTROVERSIES 

Previous  to  the  development  of  programmed  instruction  research, 
the  field  of  psychology  had  done  a  significant  amount  of  work  on  learning, 
memory,  and  other  related  areas.   However  no  unified  theory  was  generally 
available.   The  result  of  this  lack  was  that  a  considerable  argument  arose 
over  the  desirable  characteristics  which  a  teaching  machine  program  should 
have . 

As  a  result  of  their  research,  Skinner  and  his  supporters 
advocated  a  very  specific  instructional  format  in  which  small  frames 
requiring  an  overt  response  by  the  student  were  used.   Cues  to  the 
correct  answer  were  gradually  withdrawn  in  successive  frames  until  the  stu- 
dent was  on  his  own,   A  key  point  in  writing  the  program  was  to  keep  the 
error  rate  low,   If  boo  many  errors  were  made  the  reinformcement  effect 
due  to  the  constructive  response  and  the  display  of  the  correct  answer 
would  weaken  and  the  wrong  answers  might  be  remembered .   For  this  reason, 
Skinner  advocated  program  testing  and  revision  until  the  average  error 
rate  was  around  2-5%. 

Crowder,  on  the  other  hand,  used  questions  to  determine  how 
much  a  student  had  learned,  as  well  as  for  giving  him  practice  and 
reinforcement,   Errors  in  questions  would  result  in  special  remedial 
sequences  to  help  the  student  understand  why  the  error  he  had  made  was 
wrong,   Correct  answers  would  sometimes  allow  students  to  skip  instruc- 
tion which  they  already  knew.   Because  of  these  features  it  was  necessary 
to  have  a  higher  error  rate,   As  a  result,  Crowder  typically  considered 
a  15%  average  error  rate  to  be  desirable.   Skinner's  adherents,  however, 
felt  that  if  the  student  made  too  many  errors,  the  wrong  answers  might 
be  reinforced 

There  has  been  an  interesting  rebuttal  to  this  objection.   It 
has  been  argued  that  by  getting  a  question  wrong  and  then  having  to  go 
over  the  basic  principles  as  a  result,  the  student  will  end  up  with  a 
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clearer  understanding    bhe  whol«  idea  [IT,  ;,;  |.   The  supporte] 

this  idea  claim  that  LJ  a  s1  udi  I  t  responds  coi  si  I  Lmi  ,  h 

will  not  be  "set"  to  the  question  at  all  and  will  completely  dismi 

from  his  mind.   This  seems  similar  to  the  idea  of  Zeigarnik  [202],  who 

noted  a  general  tendency  toward  better  recall  of  an  unfinished  task  than  a 

finished  one, 

Another  dispute  over  intrinsic  programming  devel'  p  d  o*rcr    the 
fact  that  in  order  to  be  able  to  branch  easily,  Crowdez  used  multiple 
choice  questions  [198] •   This  resulted  in  a  problem  of  transference  of 
learning.   It  was  argued  that  in  the  real  world,  the  student  will  n 
normally  be  presented  with  a  limited  number  of  solutions  to  a  given 
problem.   Therefore  multiple  choice  questions  may  not  really  prepare  him 
for  reality. 

Another  problem  is  that  the  student  may  simply  guess  at  the 
answer ,   What  Skinner  wanted  to  reinforce  is  the  process  of  constructing 
the  answer.   By  guessing,  Skinner  claimed  that  the  student  eliminated  part 
of  the  learning  process,   He  also  felt  that  merely  seeing  the  wrong 
answers  with  the  correct  one  would  tend  to  weaken  the  learning  of  the 
correct  response . 

It  should  be  noted  that  these  objections  to  branching  methods 
can  be  partially  relieved  in  some  computer  oriented  machines  by  the  use 
of  complex  "judging"  routines.   In  this  case  a  multiple  choice  question 
is  not  necessary,   The  judging  routine  determines  the  correctness  of  a 
constructed  answer  and  determines  what  corrective  sequence,  if  any, 
should  be  entered.   The  PLATO  system,  as  an  example,  can  be  programmed 
to  classify  errors  and  transfer  to  one  of  several  sequences  depending  on 
error  type.   It  can  also  detect  misspellings  of  the  correct  answer  and 
instead  of  a  "wrong"  reply  it  will  return  a  "misspelled"  reply 

Another  area  of  disagreement  was  the  size  of  the  frame  [155, 
185].   Skinner  maintained  that  the  frame  should  be  small  since  a  response 
must  be  elicited  from  the  student  after  each  item  to  be  learned.   Since 
Crowder  was  trying  to  present  more  complete  items,  his  frames  tend  to  be 
longer.   Other  observers  felt,  that  short  frames  were  too  restrictive  for 
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certain  types  of  subject  matter  regardless  of  whether  or  not  branching  was 
used.   Smaller  frames  also  sometimes  tended  to  make  the  student  take  more  time 
to  cover  a  given  set  of  principles  '[115]  . 

Another  problem  was  whether  or  not  the  student  really  needed  to  make 
an  overt  response  in  each  frame  '[190].   It  was  felt  by  certain  groups  that  as 
long  as  the  student  made  some  sort  of  covert  response,  learning  would  be  able 
to  take  place.   It  would  also  be  much  faster  since  there  would  be  no  need  to 
write  the  answer  down.   Of  course  occasional  overt  responses  of  some  sort 
would  probably  be  necessary  if  only  to  make  sure  that  the  student  was  not 
asleep. 

In  concluding  this  section,  it  should  be  noted  that  basically  the 
linear  programming  advocates  used  an  operant  conditioning  model  of  learning 
while  the  Crowder  supporters  used  a  more  empirical  one.   They  tried  to  give 
the  student  the  information  to  be  learned  in  the  most  efficient  method  possi- 
ble and  left  it  up  to  him  to  learn  it.   There  are  several  philosophical  impli- 
cations in  these  different  techniques  (see  [993  for  a  discussion  of  linear 
programming  and  determinism) .   The  main  practical  and  theoretical  differences 
can  be  found  in  [l8,  20,    21,  25,  80,  88]. 
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III.   EXPERIMENTAL  WORK 

The  result  of  these  various  theoretical  controversies  was  a  consider- 
able increase  in  the  amount  of  research  on  teaching  machines  and  their  effec- 
tiveness.  Various  means  "were  used  to  display  teaching  machine  programs.   Due 
to  cost  considerations,  one  of  the  most  popular  was  a  programmed  text  or 
looseleaf  notebook .   These  could  be  printed  up  or  mineographed  in  quantities 
suitable  for  experimentation  at  quite  a  reasonable  price  [kk]  „   They  considted 
basically  of  sheets  of  paper  with  the  frames  printed  on  them,,   In  a  linear- 
program  the  next  frame  and  the  answer  to  the  present  one  would  be  on  the  next 
page.   In  a  branching  program,  each  multiple  choice  answer  would  direct  the 
student  to  the  page  number  of  the  frame  which  he  should  go  to  next  if  he  felt 
that  particular  answer  was  right.   In  the  branching  program  the  frames  were 
scrambled  in  a  random  order  which  caused  this  type  of  programmed  text  to  be 
called  a  "scrambled  took".   Several  publishers  have  begun  printing  texts  of 
these  types  [^1,  k-2,  52]  and  they  are  presently  one  of  the  most  popular  forms. 

One  problem  which  may  develop  with  scrambled  texts  particularly  and 
programmed  texts  in  general  is  that  they  are  bulky  and  can  become  expensive 
when  purchased  in  large  amounts  due  tc  the  fact  that  many  books  would  be 
needed  to  cover,  say,  a  year's  work  in  high  school.   Since  usually  the  student 
must  write  his  answers  in  the  book,  a.  new  text  must  be  purchased  for  each  stu- 
dent. 

Another  type  of  machine  which  is  often  used  in  experiments  is  the 
"crank"  type  in  which  the  program  is  on  a  continuous  roll  of  paper  which  the 
student  cranks  or  pulls  out  of  the  machine  as  he  goes  along.   These  are  best 
suited  for  linear  programs . 

A  third  machine  which  can  be  used  is  a  computer  generated  display. 
Due  to  its  flexibility,  an  experimenter  can  make  quick  and  easy  changes  to  the 
teaching  program  as  soon  and  as  often  as  he  wants.   The  uses  of  computers  in 
programmed  instruction  will  be  discussed  more  fully  in  Section  U. 
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In  general  there  have  been  four  basic  experiment  types  in  the  field 
of  programmed  instruction  (Pi): 

1)  Comparison  studies  between  programmed  instruction  and  conven- 
tional teaching  methods . 

2)  Parametric  studies  comparing  different  programming  methods  and 
techniques, 

3)  Exploratory  studies  investigating  new  applications  of  programmed 
instruction. 

4)  Implementation  studies  in  which  PI  is  incorporated  in  an  actual 
classroom  on  a.  continued  basis. 

The  results  of  these  different  types  of  experiments  seem  to  vary  ac- 
cording to  type. 

Experiments  of  the  first  type  were  generally  successful  in  that  al- 
most all  showed  an  increase  in  learning  and/or  a  decrease  in  time  needed  to 
study  when  programmed  instruction  was  compared  to  conventional  methods  [25, 
107,  113,  128,  129,  130,  13*1-,  141,  142,  158,  167,  1.71,  173,  176]  .   There  were 
several  exceptions  to  this  [66,    10,    126,  135,  1**3,  1*1-7,  191]  tut  such  results 
were  often  due  to  poor  experimental  conditions .   In  one  study  there  was  very 
Little  control  over  the  subject's  behavior  away  from  class  and  all  subjects 
went  to  the  same  lectures  { 1*1-3]  °   Tne  only  difference  was  that  certain  groups 
were  given  extra  time  with  programmed  materials.   Under  the  circumstances,  it 
is  not  surprising  that  no  differences  were  measured  in  the  various  groups. 

In  another  series  of  experiments  conducted  in  Great  Britain,  an 
American  program.  (Temac-First  Year  Algebra)  was  used  by  Scottish  children  who 
did  not  understand  some  of  the  terms  used.   In  still  another  experiment  two 
groups  were  used  which  had  significantly  different  mean  IQ  scores  [66] ,   An- 
other reason  for  poor  results  was  that  often  the  PI  sequences  were  much  too 
short  for  reliable  results  [13*1]  . 
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In  general,  though,  programmed  instruction  proved  itself  to  be  at 

least  equal  and  often  superior  to  more  conventional  learning  methods.  There 
has  been  one  argument  against  these  comparison  experiments,  however,  which  has 
so  far  been  difficult  to  refute:   the  suggestion  that  the  superior  performance 
of  the  students  using  programmed  materials  may  be  due  to  novelty  or  to  the 
,: Hawthorne  Effect"  .   These  critics  argue  that  once  the  novelty  wears  off  and 
students  using  programmed  learning  techniques  are  no  longer  considered  "spe- 
cial", the  advantage  of  programmed  instruction  may  lessen  or  disappear  com- 
pletely.  Although  some  experimenters  have  tried  to  lessen  these  effects  [i^l, 
1^2]  or  to  actually  measure  them  [  14-6,  170],  only  time  and  further  experimen- 
tation with  large  numbers  of  subjects  over  long  periods  of  time  will  be  able 
to  prove  or  disprove  this  objection. 

Due  to  the  generally  satisfactory  results  of  the  first  type  of  ex- 
periment, research  in  this  particular  area  has  declined  more  recently.   The 
second  type  of  experiment  is  a  different  story,  however.   The  purpose  of  these 
experiments  was  to  determine  those  programming  factors  which  led  to  the  most 
efficient  learning  for  the  student.   Unfortunately  there  are  so  many  possible 
variables,  and  their  interactions  are  so  complex,  that  it  has  been  very  diffi- 
cult to  obtain  coherent  results.   As  a  result  these  experiments  seem,  to  have 
generated  more  confusion  than  ever.   Often  when  comparing  various  techniques 
the  experiments  were  not  able  to  show  any  significant  differences  [102,  111, 
124,  125,  137,  143,  153,  16'2,  179,  I87,  192],  or  when  they  did,  the  results 
were  often  in  contradiction  with  some  earlier  experiment  [ll8,  I87] . 

Eventually  however  it  was  realised  that  there  were  several  de- 
fects in  the  research,  which  had  been  reported.   Since  nobody  was  quite  sure 
which  of  the  many  variables  in  a  teaching  machine  program  were  significant  and 
since  it  was  difficult  to  obtain  a,  sufficient  population  of  suitable  subjects 
in  order  to  test  all  variables,  mistakes  were  often  made  in  the  choice  of  per- 
tinent variables. 
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Some  investigators,  in  an  effort  to  control  as  much  of  the  experiment 
as  possible  and  to  test  only  a  small  number  of  variables  at  a  time,  tried  to 
write  one  program  and  then  modify  it  slightly  in  order  to  compare  different 
programming  variables  [e.g.,  12 k,    lUO]  .   This  is  a  reasonably  sound  procedure, 
providing  that  the  variables  being  measured  and  the  variables  held  fixed  have 
independent  effects  on  learning.  Unfortunately  this  is  generally  not  the  case 
in  programmed  instruction  and  the  net  effect  of  this  type  of  experiment  is  to 
show  little  or  no  effect  on  learning.   What  is  more  likely  important  is  the 
variation  of  combinations  of  variables  so  that  if  two  programming  techniques 
3,re  to  be  compared,  the  amount  of  learning  which  is  produced  by  each  should 
first  be  optimized  with  respect  to  all  of  the  other  variables.   Up  to  now, 
however,  there  has  not  been  enough  time,  money,  or  subjects  to  do  anything  like 
this . 

Another  problem  with  programmed  instruction  research  was  the  quality 
of  the  programs  used.   It  often  turned  out  that  the  programs  which  were  used 
in  the  experimental  research  were  not  written  in  such  a  manner  as  to  enable  a 
distinction  to  be  made  between  two  techniques.   For  example,  in  one  test  be- 
tween linear  and  branching  programs  the  lack  of  a  difference  in  the  amount  of 
learning  was  explained  (by  the  experimenter)  as  being  due  to  the  unappropri- 
ateness  of  the  remedial  branching  sequences  [187] °   Since  the  remedial  se- 
quences vere  not  very  good,  the  students  with  branching  learned  no  more  than 
those  without  it. 

In  support  of  this  observation,  J.  L.  Evans,  et  al°  [12k]    observed 
that  the  relavence  of  any  particular  program  variable  in  a  frame  was  inversely 
related  to  the  probability  of  a  correct  response  by  the  subject  on  that  frame 
J.  G-   Holland  has  developed  a  measure  for  the  criticality  of  the  stimulus  ma- 
terial in  a  frame  [3,  ^3] ■   This  measure  is  called  the  black-out  ratio  and 
represents  the  maximum  ratio  of  the  amount  of  material  in  a  frame  which  can 
be  removed  without  affecting  the  student  error  rate,  to  the  total  amount  of 
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material  in  the  frame .  He  discovered  that  programs  which  had  high  black-out 
ratios,  i.e.,  those  which  contained  much  non-critical  material,  when  used  in 
riments  to  compare  overt  and  covert  responding,  constructed  response  vs. 
multi-choice  response,  etc.,  showed  no  significant  differences  between  the 
variables  being  testedo   Conversely  programs  with  low  black-out  ratios  were 
able  to  show  significant  differences. 

Sometimes  the  results  of  a  programmed  learning  experiment  were  af- 
fected by  a  lack  of  familiarity  on  the  part  of  the  subjects.   In  one  case 
Bivens  and  Cambell  [102,  110]  noticed  that  in  a  comparison  between  self -di- 
rected and  non-self -directed  learning,  students  who  have  become  accustomed  to 
the  lecture  method  in  school  do  not  really  know  how  to  direct  their  own 
studying.  However  after  being  given  a  coaching  session  in  self -study  methods 
they  were  superior  to  a  group  using  linear  programmed  instruction. 

J.  E.  Coulson,  et  al.  [119]  have  noticed  this  same  factor  in  that 
when  they  gave  students  the  option  of  controlling  their  own  branching  in  a 
teaching  program,  they  rarely  took  advantage  of  it. 

In  addition  to  this  there  were  many  other  more  standard  types  of 
problems.   It  was  often  difficult  to  decide  how  to,  measure  the  increase  in 
learning  [68].   Poor  pre-  and  post-testing  of  the  subjects  would  often  lead 
to  fairly  useless  results  [l6U]  .   In  some  cases,  where  the  experimenter,;  were 
using  students  in  college  classes  there  was  often  very  poor  control  over  the 
kinds  of  learning  the  students  acquired  when  away  from  the  teaching  programs 
[96,  l'-K3]  •   Then  too,  the  students  motivation  often  varied  from  group  to  group 
L95J  • 

To  conclude  this  discussion  of  the  type  two  experiments,  there  is 
now  at  least ' a  body  of  knowledge  which  can  be  used  to  devise  new  experiments, 
although  there  is  still  much  which  is  unknown  about  the  factors  which  cause  a 
program  to  teach  efficiently.    In  the  future  more  experiments  will  be  needed 
which  try  to  impose  some  order  upon  this  body  of  knowledge.  One  example  of 
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this  type  of  experiment  is  Leigh's  results  [15,1]  wnich  were  able  to  experimental- 
ly show  "why  experiments  by  other  researchers  [104,  124,  133,  1^2,  148,  149,  198, 
201]  had  been  so  confusing.   3y  discovering  that  the  effect  of  overt  as  opposed 
to  covert  responses  varied  with  the  type  of  material  being  taught,  Leigh  was 
able  to  explain  the  often-conflicting  results  of  previous  experiments „ 

There  are  various  examples  of  the  third  type  of  experiment  [108,  112, 
138,  139.  157,  160,  180,  184,  185,  189] •   This  type  consists  of  various  explor- 
atory experiments  which  are  aimed  at  finding  qualitative  results  from  various 
extensions  of  programmed  instruction.  Most  of  the  computer  experiments  which 
will  be  described  in  Section  IV  are  of  type  three.  Other  examples  of  this  type 
of  experiment  include  one  which  tested  the  effect  of  problem  setting  questions 
in  a  "conventional"  linear  program  [137] >  investigations  into  self -directed 
programmed  instruction  [102,  110],  and  various  investigations  using  adaptive 
programming  [154,  l6l,  188,  193] ■ 

The  type  four  experiments,  consist  of  implementations  of  what  is  al- 
ready known.   Their  orientation  is  more  practical  than  theoretical,  for  by  the 
time  a  school  system  makes  up  its  mind  to  use  programmed  learning  techniques, 
it  is  no  longer  interested  in  detailed  scores,  learning  curves,  etc.,  but  in- 
stead wants  to  know  the  effects  on  teachers'  loads,  how  big  the  classrooms 
should  be,  the  grading  system  to  be  used,  and  other  mundane  affairs  which  need 
to  be  known  to  make  a  practical  system. 

In  general  there  have  not  been  too  many  large  scale  implementations 
of  programmed  instruction.  A  primary  reason  for  this  is  the  lack  of  a  coordi- 
nated series  of  programs  spanning  several  grade  levels.   Those  implementations 
that  do  exist,  however,  can  be  found  in  industry  [lOf,  l45,  163,  168,  l69]j> 
the  government  [121,  166],  and  the  educational  system  [9h,   ll4,  120,  122,  136, 
182,  194] .   In  general  these  implementations  have  shown  the  following" 
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1)  Programmed  instruction  techniques  are  affective  teaching  methods. 

2)  Teachers  must  be  instructed  in  the  use  of  PI  before  a  PI  program 
is  begun  at  school. 

3)  Although  often  adaptable  to  individual  student  difference s} 
teaching  machine  programs  can  be  very  inflexible  when  responding 
to  curriculum  changes, 

h)     At  the  present  time  there  is  a  chronic  lack  of  good  programs, 

especially  series  of  programs  covering  one  subject  through  several 
grade  levels . 

In  the  following  section  the  application  of  computers  to  programmed 
instruction  will  be  described. 
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IV.   THE  APPLICATION  OF  COMPUTERS  TO  PROGRAMMED  INSTRUCTION 

When  Skinner  produced  his  first  teaching  machine,  one  of  its  impor- 
tant characteristics  was  its  simplicity  and  low  cost.  However,  this  simplicity 
was  also  one  of  its  major  defects  since  the  type  of  interaction  between  the 
student  and  the  machine  was  very  limited.  When  Crowder  introduced  branching 
programs,  he  was  able  to  deal  with  students  in  a  more  flexible  way  with  a 
slight  increase  in  complexity.  With  the  advent  of  computer  usage  for  automated 
teaching  systems,  an  even  greater  amount  of  flexibility  became  possible  [1,  67] 
but  at  a  very  high  cost.  As  a  result  very  few  people  could  afford  to  investi- 
gate the  computer  applications  of  teaching  machines  unless  they  already  had  a 
computer  available  for  other  applications.   This  has  meant  that  most  of  the 
work  in  computer-assisted  instruction  ( CAI)  has  been  done  by  computer-oriented 
people  and,  as  a  result,  the  research  in  CAI  has  maintained  a  completely  dif- 
ferent flavor  from  other  PI  research.   The  CAI  work  has  generally  been  carried 
out  on  a  very  pragmatic  basis.   This  may  be  due  to  the  fact  that  most  of  the 
people  in  this  area  were  originally  educated  as  engineers  and  not  psychologists 
At  any  rate  these  practitioners  tend  to  think  (and  publish)  in  terms  of  appli- 
cations ard  not  in  theory. 

The  first  CAI  experiments  were,  done  on  free-standing  machines  where 
only  one  student  could  use  the  machine  at  a  time.  This  was  fantastically  ex- 
pensive, however.  Licklider,  of  Bolt,  Beranek  and  Newman,  estimates  a  cost  of 
$3?/^r  -or  one  student  using  a  PDP-1  computer  [156] °  When  time  sharing  began 
to  develop,  multiple-console  systems  were  quickly  developed  so  that  many  stu- 
dents could  use  the  same  computer  at  the  same  time. 

Although  time-sharing  allows  the  costs  to  be  spread  over  many  stu- 
dents, it  aggravates  another  problem.   In  this  sort  of  system  each  person  must 
be  equipped  with  their  own  input-output  console.  An  electric  typewriter  is 
usually  a.  good  input  device.  Maximumly  effective  learning  however,  requires 
that  there  must  not  be  a  long  delay  when  material  is  being  presented  to  the 
student.   This  means  that  the  output  must  be  displayed  by  a  fairly  high  speed 
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device  such  as  a  slide  projector,  TV  screen,  or  similar  instrument,  Unfortun- 
ately a  slide  projector  limits  versatility  since  it  must  be  pre-loaded  with 
slides  and  cannot  be  used  to  display  computer -generated  information-  A  TV  dis- 
play costs  more  money  than  can  really  be  afforded,  especially  after  the  neces- 
sary memory  circuitry  is  added. 

For  the  present  time  it  appears  that  the  average  cost  of  such  a  TV- 
typewriter  console  is  on  the  order  of  $10,000.   D«  L.  Bitzer  [5]  has  estimated 
that  an  economical  automated  instruction  system  (on  the  order  of  $.25/hr/ stu- 
dent) could  be  built  if  the  cost  of  the  consoles  can  be  cut  to  $1200.   This  in- 
volves timesharing  4000  students  on  a  CDC  6600  computer,  the  largest  and  fastest 
computer  available  today.  A  recent  breakthrough  in  panel  display  devices  [6] 
seems  to  indicate,  according  to  Bitzer,  that  a  reliable  console  with  the  needed 
resolution  and  with  an  inherent  memory  capability,  can  indeed  be  built  for  this 
price  within  3  years. 

With  the  advent  of  more  sophisticated  computer  programming  the  amount 
and  type  of  control  which  the  student  may  have   over  the  instruction  sequence 
has  engendered  extensive  discussion  and  experimentation  [29,  102] .   In  a  basic 
Skinner  or  Crowder  program  the  student  has  little  conscious  control  over  the 
process,  as  he  is  only  answering  questions  which  are  put  before  him  by  the 
teacher -programmer .  A  more  recent  technique  using  computers  involves  a  "so- 
cratic"  form  of  teaching.   Swets  and  Feurzeig  [85;  127 ]  describe  a  program 
which  teaches  medical  diagnosis  by  setting  up  a  basic  case  history  and  then 
requires  the  student  to  request  the  proper  information,  to  make  the  diagnosis. 
The  student  may  ask  for  laboratory  data  or  initiate  a  more  thorough  "physical 
examination" .   If  he  asks  for  irrelevant  information  or  makes  a  stupid  diag- 
nosis he  is  properly  corrected.   The  complete  interaction  takes  place  in  a 
conversational  mode. 

Another  example  of  this  type  of  teaching  is  the  PLATO  inquiry  logic 
[99>  100] .   In  this  mode  of  operation  the  computer  shows  the  student  an  exper- 
iment on  the  TV  screen.   The  student  must  then  answer  a  series  of  questions. 
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In  answering  them,  the  student  can  ask  various  questions  of  the  computer  and 
request  the  results  of  an  experiment  if  some  initial  condition  had  been  dif- 
ferent-  In  this  case  the  experiment  is  rerun  using  the  new  conditions. 

In  some  experiments  the  student  has  been  given  sufficient  control 
over  the  learning  process  that  the  computer  becomes  merely  a  tool  and  is  no 
longer  a  teacher.   This  was  done  in  one  Boston  school  in  which  students  were 
given  access  to  a  time -shared  computer  with  a  very  simple  programming  language 
and  were  allowed  to  use  it  to  solve  any  type  of  math  problems  they  wanted  [175] 

In  a  student- controlled  environment  other  than  this  extreme  case,, 
however,  there  are  some  severe  problems  concerned  with  understanding  the  stu- 
dent input  information.   The  computer  must  be  able  to  interact  in  a  conversa- 
tional mode  with  the  student  and  to  some  extent  at  least  be  able  to  "under- 
stand"  what  he  is  saying.   This  area  has  been  increasingly  developed  lately, 
not  only  from  the  standpoint  of  teaching  machine  programs  but  also  with  man- 
computer  interactions  in  general  [_'( ,   8,  38,  6!4,  79>  84]  .  As  of  yet  no  com- 
pletely general  solutions  to  this  problem  have  been  found. 

There  are  other  teaching  situations  however,  which  can  be  considered 
to  be  student  controlled  which  are  not  terribly  difficult  to  achieve  right 

One  is  the  educational  game,  Leonard  and  Wing  describe  a  computer  pro- 
gram which  is  used  to  teach  economics  to  children  [152,  200] .   The  student  is 
put  in  a  hypothetical  situation  such  as  being  king  of  a  mesopatanim  kingdom. 
As  such,  he  must  make  economic  decisions  such  as  how  much  grain  to  plant  and 
ho-  much  to  feed  to  his  subjects.   The  results  of  his  decisions  are  calculated 
and  given  back  to  him  (i.e.,  half  your  people  have  starved  to  death).  At 
first  the  decisions  and  their  results  are  fairly  obvious  in  order  to  teach  the 

ident  the  economic  principle  involved.  As  time  goes  by  the  problems  become 
more  and  more  complicated  until  the  student  is  making  a  variety  of  complex  de- 
Lons  and  in  the  process  learning  principles  of  economics. 
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V.   FUTURE  PROSPECTS 

At  the  present  time  the  future  of  programmed  instruction  appears 
fairly  hopeful .   There  are  many  advantages  in  using  PI,  among  them: 

1)  Providing  immediate  feedback  to  the  student. 

2)  Allowing  the  teacher  to  determine  exactly  what  part  of  the  in- 
struction sequence  needs  improving  when  the  students  do  not 
learn  a  specific  item. 

3)  Providing  more  and  better  education  for  both   advanced  and  re- 
tarded students  by  allowing  students  to  go  at  their  own  pace 
with  program  sequences  adjusted  to  their  own  level = 

h)      Reduction  of  the  drudgery  work  for  teachers. 

In  the  next  few  years  research  will  continue  to  investigate  the 
basic  programming  principles  and  search  for  less  expensive  hardware.   In  addi- 
tion to  this,  however,  a  great  deal  of  work  will  need  to  be  done  in  the  school 
system  in  order  to  prepare  for  the  large-scale  changes  of  the  next  few  years. 
Not  all  of  these  changes,  of  course,  will  be  due  to  programmed  instruction  but 
PI  will  probably  be  involved  in  some  major  changes  to  come. 

For  one  thing,  although  programmed  lee.rr.ing  will  definitely  not 
eliminate  the  teacher  from  the  classroom,  it  will  significantly  upgrade  the 
basic  teaching  task.  L.  F.  Carter  [11]  mentions  seven  basic  goals  of  the 
American  school  system,  including  in  them  teaching  socialization,  developing 
creativity,  and  encouraging  physical  development  in  addition  to  teaching  basic 
"fact"  subjects.   If  teachers  are  able  to  use  machine  programs  to  teach  the 
more  routine  fact  learning  in  the  school  curriculum,  the  teacher  will  become 
more  free  to  develop  the  students'  thinking  processes,  discuss  the  relation- 
ships between  groups  of  facts,  and  stimulate  creativity. 
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Another  effect  of  extensive  use  of  programmed  instruction  techniques 
beaching  will  be  to  put  an  even  greater  strain  on  our  school  systems'  at- 
tempts to  reta.in  "che  chronological  grade  format  presently  being  used.   Since 
students  will  tend  to  move  at  their  own  pace,  it  is  obvious  that  grouping  stu- 
dents into  classes  on  the  basis  of  chronological  age  will  become  very  diffi- 
cult.  There  have  indeed  been  experiments  in  trying  to  accomplish  group  pacing 
of  students  using  programmed  instruction  [I31>  165],  but  these  seem  to  work 
best  when  the  groups  were  selected  on  the  basis  of  IQ,  or  achievement  test 
scores  which  means  that  the  chronological  age  was  not  important  .   It  appears 
therefore  that  the  socialogical  implications  of  grouping  students  according  to 
mental  age  and  interests,  instead  of  chronological  age,  will  have  to  be  faced. 

Systems  have  been  suggested  [39]  in  which  students  spend  part  of 
their  time  on  "directed"  work  and  part  of  their  time  exploring  subjects  of 
their  own  choosing  at  whatever  level  of  difficulty  they  can  master.   This  in- 
dividualized instruction  could  be  done  with  programmed  instruction,  special 
lectures  or  discussion  groups  or  a  combination  of  these.  Note  that  with  indi- 
vidual programmed  instruction  the  level  of  knowledge  of  a  particular  group  of 
students  can  be  brought  to  a  specified  level  before  a  specific  lecture  or  dis- 
cussion so  that  the  students  will  all  be  able  to  get  maximum  benefit  from  the 
activity. 

Another  administrative  problem  which  will  probably  come  under  review 
is  the  problem  of  grading.  With  the  increase  in  individualization  which  a 
large-scale  integrated  use  of  PI  will  bring,  the  problem  of  measuring  relative 
achievement  will  increase.   Some  advocates  of  PI  have  gone  so  far  as  to  say 
that  the  extensive  use  of  PI  will  eliminate  the  need  for  grades  since  a  good 
teaching  machine  program  will  teach  the  required  subject  matter  to  everyone 
who  can  complete  the  program.   Tne  fact  remains  however  that  different  stu- 
bs can  and  do  learn  different  amounts  from  the  same  program,  so  some  sort 
of  grading  scheme  may  still  be  desirable. 
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For  the  immediate  future,  Programmed  Instruction  presents  a  differ- 
ent set  of  problems.  In  the  research  area  more  work  must  be  done  in  ascer- 
taining the  exact  characteristics  which  optimize  learning  in  a  teaching  pro- 
gram.  This  will  take  a  period  of  extensive  testing.   There  is  also  a  need  for 
a  low-cost  flexible  machine  which  can  be  easily  updated  as  new  research  devel- 
ops new  techniques.  Finally  a  large  number  of  programs  must  be  written  and 
pre-tested  before  being  used  in  the  school  system.   This  is  especially  true 
since  the  majority  of  programs  and  machines  which  were  produced  in  the  early 
60's  were  not  very  useful  for  practical,  large-scale  teaching. 

The  main  immediate  goal  for  the  teachers  and  administrators  in  the 
school  system  will  probably  be  twofold.  First  many  more  teachers  and  parents 
will  be  taught  the  basic  ideas  and  principles  of  programmed  instruction. 
Secondly,  teachers  themselves  will  need  to  enter  the  research  activities  in 
programmed  instruction  and  determine  more  precisely  which  areas  of  the  curric- 
ulum will  benefit  most  from  this  new  technique.  As  this  is  done  implementing 
programmed  instruction  into  the  school  system  will  become  practical  and  bene- 
ficial. 
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Short  note  on  a  German  language  program  (73)  which  used 
partially  translated  stories  by  E„  A.  Poe, 

75.  Schramm,  W.  A.,  Programmed  Instruction  -  Today  and  Tomorrow,  Fund  for 

the  Advancement  of  Education,  November  1962. 

Reviews  history  and  all  experimental  work  in  PI  up  to  1962 
but  no  reference  list. 


76.  Schramm,  W.A,,  The  Research  on  Programmed  Instruction;   An  Annotated 

Bibliography.,  Office  of  Education,  Bulletin  No,  35,  U.S.  Dept. 
H.E.W. ,  Washington,  D.C.  1964. 

77,  Silberman,  H.F.,  "Self-Teaching  Devices  and  Programmed  Materials", 

Review  of  Educational  Research,  Vol.  32,  No,  2,  1962,  pps.  179-193, 
Good  review  of  literature  on  PI  and  good  bibliography. 

78.  Simmons,  R,F.  ,  "Answering  English  Questions  by  Computer:   A  Survey", 

Comm.  of  the  ACM.  Vol.  8,  No.  1,  January  1965,  pps .  53-70, 

79,  Skinner,  B.  F,,  "The  Science  of  Learning  and  the  Art  of  Teaching", 

Harvard  Educational  Review,  Vol.  24,  1954,  pps.  86-97-   Also 

in  Smith  (8l). 

One  of  the  first  important  articles  on  teaching  machines, 
Discusses  lab  work  on  learning  and  conditioning  and  compares 
this  with  current  educational  systems. 


80.  Skinner,  B.F.,  ,  "Why  We  Need  Teaching  Machines",  Harvard  Educational 

Review.  Vol   31,  No,  4',  Fall  196l,  pps .  377-398.   Also  in  de  Grazia 

and  Sohn 

Very  complete  argument  for  linearly  programmed  teaching  machines. 
Presents  the  arguments  against  branching  programming  and  the 
"feedback"  principles, 

81.  Smith,  W.I.  and  Moore,  J.W.,  Programmed  Learning:   Theory  and  Research, 

D.  Van  Nostrand  Co  ,  Inc.,  Princeton,  N,J\  ,  1962. 

82.  Stolurow,  L.M  ,  Teaching  by  Machine,  Cooperative  Research  Monography 

No.  6,  U.S.  Dept  HEW,  CLE.-3U010 ,  196l. 

Excellent  oA^erview  of  field,   Describes  machines  in  existence, 
learning  theory,  programming  techniques,  and  research  up  to 
i960,   A  comprehensive  set  of  references  is  included, 

83=   Stolurow,  L.M. ,  "Principles  for  Programming  Learning  Materials  in  Self- 
Instructional  Devices  for  Mentally  Retarded  Children"  Final  Report 
Cooperative  Research  Project  No,  66l ,  SAE  8370,  U  of  111.,  1963. 
Contains  5  reports  and  an  extensive  bibliography.   Concerned 
with  the  effect  of  cueing  and  stimulus  control, 

8U.   Stone,  P.J. ,  Bales,  R.F. ,  Namenwirth,  J.Z.,  and  Ogilvie,  D.M. ,  "The 
General  Inquirer:   A  Computer  System  for  Content  Analysis  and 
Retrieval  Based  on  the  Sen.     as  a  Unit  of  Information", 
Behavioral  Science,  Vol,  '( ,  October  1962,  pps,,  i+8'4-498. 


85.  Swets ,  J, A,  and  Feurzeig,  W,,  "Computer-Aided  Instruction",  Science, 

Vol.  150,  October  1965,  pps   572-576,, 

Describes  conversational  mode  teaching  using  a  Socratic-type 
method,   Discusses  how  to  reduce  rich  materials  to  a  step- 
by-ster.  exposi*  ion 

86.  Taber,  J.I,  .  Glaser,  R.,  and  Schaefer,  H..H  ,  Learning  and  Programmed 

Inst  r-;c  ~  .yen ,  Addi s  on-We s  1  ey ,  Re adi ng ,  Mas s  „  ,  1 96 5  . 

Discusses         es  and  pn    Lures  in  PI«   Reviews  definitions 

I  stim  lus  ,  response,  reinf<  rcement,  extinction,  etc,  as  they 
P  e 

87.  Teal,  G.E.  (ed.  ) ,  Pr-1^-"^mmed  Ins  t  ro-t  ion  in  Industry  and  Education,  Public 

Service  Research.  Inc.,  Stan     ,  Conn,  1963. 

Ti     Lp1  of  a  '.962  institute  on  PI.   Covers  the  field  from 
learning  theory  +;  programming  methods  tc  evaluation.   Hard 
efei   '  bi   luse  of  dialog  format, 

88 ,,   Thomas,  C  A,  end  Davies ,  I.K. ,  Programmed  Learning  m  Perspective, 
The  Adelpbi  Press,  Barking,  Essex,  G.B. ,  1963: 

Approaches  PI  from  systems  viewpoint.   Discusses  program 

Lng  +he  ruleg  system  -  rule  identification,  matrix 
,  diagrams  and  frame  writing.   Two  complete 
examples 


89.  Thornhill,  P.,  "What  Future  has  Programmed  Learning  in  British  Schools", 

from  Unwin  (91),  1967,  pps .  75-81 . 

Overview  of  British  PI  field  from  a  publisher's  viewpoint. 
Presents  preliminary  results  of  a  1964-65  survey.   Suggests 
that  a  vicious  circle  may  develop  in  which  a  lack  of  good 
integrated  program  series  will  stifle  demand  which,  in  turn 
will  restrict  production. 

90.  Tompkins,  H.E.,  "Programmed  Instruction",  Advances  in  Computers .  Vol-  4, 

F.L.  Alt  and  M.  Rubmoff  (eds),  Academic  Press,  New  York,  pps,  158-l6l 
Quick  review  of  the  use  of  computers  in  PI.   Contains  a  short 
but  good  bibliography  with  comments. 

91.  Unwin,  D.  and  Leedham,  J.,  Aspects  of  Educational  Technology,  Methuen 

and  Co,  ,  Ltd.  ,  London,  'SCj '. 

Contains  43  papers  presented  at  a  1966  conference  in 
Loughborough,  England.   Concerned  with  production  and  use 
of  PI  in  Britain  and  Europe, 

92.  Winn,  S. ,  "Programmed  Instruction:   Panacea  or  Plague?",  Chicago 

Schools  Journal,  Vol.  43,  No.  3,  December  196l,  pps.  131-133, 

93.  Wohlwill,  J.F.,  "The  Teaching  Machine:   Psychology's  New  Hobbyhorse", 

Teacher's  College  Record,  Vol.  64 ,  No.  2,  Nov,  1962,  pps.  139-146. 

Experimental  Reports: 

94.  Barkus ,  D. ,  Hayman,  J.L.,  and  Johnson,  J.T.,  "Programming  Instruction 

in  Elementary  Spanish",  from  de  Grazia  and  Schn  ( 22)  ,1964  ,pps „  226-232 
Discusses  Denver  School  System  experiment  with  programmed 
learning  in  Spanish  (see  1.82)  and  presents  some  of  the  data, 

95.  Bartz,  W.H„  and  Darby,  C.L.,  "A  Study  of  Supervised  and  Ncnsupervised 

Programmed  Instruction  in  a  University  Setting",  Journal  of  Ed. 
Research.  Vol.  58,  No.  5.  January  1965 

96.  Bartz,  W.H.  and  Darby,  C.L. ,  "The  Effect  of  a  Programmed  Textbook  on 

Achievement  Under  Three  Techniques  of  Instruction",  Journal  of 
Experimental  Education,  Vol.  34,  No=  3,  pps.  46-52. 

97.  Beane,  D,G,,  "A  Comparison  of  Linear  and  Branching  Techniques  of 

Programmed  Instruction  in  Plane  Geometry",  Journal  of  Educational 
Research.  Vol,,  58,  No.  7,  March  1965 ,  pps.  318-326, 

98.  Bitzer,  D.L.  et .  al .  "The  PLATO  System:   Current  Research  and 

Developments",  IEEE  Transactions  on  Human  Factors  in  Electronics, 
Vol.  HFE  8,  No-  25  June  1967 ,  pps.  64-70. 

99-   Bitzer,  D.L,  Lyman,  and  Suchman,  "REPLAB  -  A  Study  in  Scientific 

Inquiry  Using  the  PLATO  System",  U,  of  HI.,  Coordinated  Science 
Lab.  Report  R-260 ,  December  1965. 


100.  Bitzer,  D.L. ,  et.al.,  "The  Uses  of  PLATO:   A  Computer  Controlled 

Teaching  System",  U.  of  111,  Coordinated  Science  Lab.  Report 
R-268,  October  1965. 

101.  Bitzer,  D.L.  et.  al. ,  "PLATO  II:   A  Multiple-Student ,  Computer- 

Controlled,  Automatic  Teaching  Device",  from  Coulson  (l6)s  1962, 
pps.  205-206. 

102.  Bivens,  L.  W.  Cambell,  V.N- ,  and  Terry,  D.F.,  "Self  Direction  in 

Programmed  Instruction:   Effects  on  Learning  in  Low-Ability 
Students"  American  Institute  for  Research  Report  AIR-D10 -T163-TR , 
July  1963. 

103.  Braunfeld,  P.G.  ,  "Problems  and  Prospects  of  Teaching  with  a  Computer", 

Journal  of  Educational  Psychology,  Vol.  55 >  No.  ht    1964,  pps  ;  201- 
211. 

Describes  one  of  several  informal  experiments  using  the  PLATO 

II  system. 

10U,  Briggs,  L.J.  et.al.,  "Experimental  Results  Regarding  Form  of  Response, 
Size  of  Step,  and  Individual  Differences  in  Automated  Programs", 
from  Coulson  (l6),  1962,  pps .  86-98, 

105^   Briggs,  L.J.  et.al.,  "Investigations  of  Thinking  via  Self-Instructional 
Programs",  American  Institute  for  Research,  AIR-DU1-6/6U-FR ,  June 
1964. 

Describes  in  detail  two  experiments  (106 ,  111)  with  varying 
formats  and  presentation  techniques. 

106,   Brooks,  L.O.,  "Laboratory  Study  of  Individual  Performances  on  Programmed 
Learning  Tasks",  from  Briggs  (105),  196U,  pps.  32-69 0 

Investigates  latency  time  as  a  measure  of  frame  difficulty, 

107  Bruce,  J.S.  "Applied  Research  on  Programmed  Instruction  a"c  the 

Eastman  Kodak  Company",  from  Hughes  (^5),  1963,  pps,  59-110 

Describes  integrated  use  of  PI  in  Kodak  training  programs. 

108  Camb     v.N  ,  "Studies  of  Bypassing  as  a  Way  of  Adapting  Self- 

Ir.str actional  Programs  to  Individual  Differences",  American 
Institute  lor   Research,  AIR-CU.l-5.'62-FR,  May  1962.   Also  reported 
in  J,  of  Ed.  Psy.,  Vol.  5^,  No,  6,  1963,  pps,  337-3^5- 

109-   Cambell,  V.N.  et .  al.  ,  "Effects  of  Mathematical  Ability,  Pretraimng, 
and  Interest  on  Self-Direction  in  Programmed  Instruction", 
American  Institute  fox  Research,  AIR-D10-10/63-TR,  Oct,.  1963. 

110,   Cambell,  V.N  ,  "Self-D     Lon  and  Programmed  Instruction  for  Five 
Different  Types  of  Learning  Objectives",  American  Institute 
for  Research,  AIR-D10-12/63-TR(b) ,  December  1963. 


111.  Cambell,  V.N. ,  "Learning  to  Solve  Problems:   A  Classroom  Experiment", 

from  Briggs  (105),  196U ,  pps .  12-31. 

Investigated  the  effects  of  various  types  of  reviews  after 
studying  a  PI  geography  sequence,  on  the  ability  to  solve 
problems  in  geography. 

112.  Cambell,  V.N.,  and  Chapman,  M. ,  "Degree  of  Student  Control  over 

Programmed  Instruction:  Long  Term  Cummulative  Effects  on  Problem 
Solving  and  Transfer",  American  Institute  for  Research,  AIR-E20- 
12/65-TR,  Dec.  1965 

113.  Cassel,  R.N.  and  Ullom,  W.L.,  "Preliminary  Evaluation  of  Programmed 

Instruction  with  Students  of  High  Ability",  Psychological  Reports, 
Vol,  10,  No.  1,  February  1962,  pps.  223-228, 

11^.   Clarke,  J.,  "The  Development  and  Use  of  Linear  Programmed  Instruction 
in  a  Rural  Primary  School",  from  Unwin  (91),  1967,  pps .  UI-U9. 

115.  Coulson,  J.E.  and  Silberman,  H.F.  ,  "Effects  of  Three  Variables  in  a 

Teaching  Machine",  Journal  of  Educational  Psychology,  Vol,.  51,  No,  3, 

I960,  pps.  135-1^3. 

Investigates  the  effects  of  multiple  choice  vs,  constructed 
response,  size  of  the  frame,  and  fixed  sequences  vs.  branching 
sequences . 

116.  Coulson,  J.E.  and  Silberman,  H.F.  "Automated  Teaching  and  Individual 

Differences",  from  Smith  (8l),  1962,  pps .  207-217.   Also  in 
Audio-Visual  Communications  Review,  Vol.  9,  196l,  pps.  5-15 » 

Describes  the  System  Development  Corp,  teaching  system 

using  a  Bendix  G-I5  computer. 

117.  Coulson,  J.E. ,  "A  Computer-Based  Laboratory  for  Research  and  Develop- 

ment in  Education"  from  Coulson  (l6)9  1962,  pps.  191-20U. 

Describes  CLASS,  an  integrated  approach  to  computer-aided 
education, 

118.  Coulson,  J.E,  et ,  al. ,  "Non-Program  Variables  in  the  Appliance  of 

Programmed  Instruction",  System  Development  Corp,,  TM-2176/201/00 , 

July  1965. 

Reports  on  two  experiments  in  which  the  teacher  augmented 
ordinary  PI  or  the  students  controlled  their  own  branching. 

119.  Coulson,  J.E.,  "Effects  of  Branching  in  a  Computer  Controlled  Auto- 

Instructional  Device",  Journal  of  Applied  Psychology,  Vol,  U6, 
No.  6,  1962,  pps.  389-392, 

120.  Cowan,  P„J,,  "Autoinstructional  Materials  in  Teaching  Physics  in 

Small  Schools",  J,  of  Exp,  Ed. ,  Vol.  36,  No,  1,  Fall  1967,  pps .  k6- 
50. 


121=   Davies,  I.K.,  "Programmed  Learning  in  the  United  States  Air  Force", 

Programmed  Learning,  Vol.  1,  No.  3,  November  196U,  pps .  113-123. 
Describes  a  two-phase  development  program  to  implement 
PI  in  the  USAF  (1961-1963). 

122.  Edling,  J.V.,  "Programmed  Instruction  in  a  'Continuous  Progress' 

School  -  Provo,  Utah",  from  Schramm  (l8l),  1964,  pps.  66-94. 
Reports  on  two  years  of  use  of  PI  at  the  Brigham  Young 
University  Laboratory  School,  a  "non-paced"  school. 

123.  Englund,  D.E.  and  Estavan,  D.P. ,  "CLASS  -  The  Automated  Classroom", 

Computer  Applications  196l-Proeeedings  of  the  196l  Computer 
Applications  Symposium,  R.S.  Hollitch  and  B,  Mittman  (eds.), 
The  MacMillan  Co.,  New  York,  1961,  pps.  177-188. 

124.  Evans,  J.L.,  Glaser,  R. ,  and  Homme,  L.E,  ,  "An  Investigation  of 

'Teaching  Machine'  Variables  Using  Learning  Programs  in  Symbolic 
Logic",  from  Glaser  (132),  1961,  pps.  31-81. 

125.  Feldhusen,  J.F.  and  Birt,  A.,  "A  Study  of  Nine  Methods  of  Presentation 

of  Programmed  Learning  Material",  J.  of  Educational  Res.,  Vol. 
55,  1962,  pps,  h6l-k66. 

126.  Feldman,  M.E. ,  "Learning  by  Programmed  and  Text  Format  at  Three 

Levels  of  Difficulty",  Journal  of  Educational  Psychology, 
Vol.  56,  No.  3,  pps.  133-139. 

127.  Feurzeig,  W. ,  et.al.,  "Computer-Aided  Teaching  in  Medical  Diagnosis", 

J.  of  Medical  Ed.  ,  Vol.  39,  August  196U,  pps.  746-754. 

128.  Finelli,  CM.,  "The  Application  of  Programmed  Learning  Techniques  to 

Selected  Material  in  a  Professional  Curriculum- A  Study  of 
Retention  Effects",  from  Glaser  (132),  1961,  pps.  115-133. 

129.  Flynn,  J.T.,  "The  Influence  of  Programmed  Instruction  Upon  Learning 

in  Educational  Psychology"  Journal  of  Educational  Research, 
Vol.  59,  Nco  9,  1966,  pps.  387-391 . 

130.  Friedman,  M.I.,  "The  Effectiveness  of  Machine  Instruction  in  the 

Teaching  of  Second  and  Third  Grade  Spelling" ,  J.  of  Educ .  Res. , 
Vol.  60 ,  No,  8,  April  1967,  pps.  366-369. 

Results  tend  to  show  PI  better  ax   3rd  grade  level  but  not 

at  2nd  grade  level, 

131.  Frye,  C.H.,  "Group  vs.  Individual  Pacing  in  Programmed  Instruction", 

U.S.  Dept.  of  H.E.W. ,  Report  for  Nat.  Def.  Ed.  Act  of  1958, 
Grant  No.  7-47-0000-158. 


132.  Glaser,  R. ,  Investigations  of  the  Characteristics  of  Programmed 

Learning  Sequences.  Programmed  Learning  Laboratory,  U.  of  Pitt., 

October  1961. 

Reports  on  a  series  of  experiments  using  linear  programs 
(73,  124,  128,  138,  179).   Developes  the  ruleg  system  of 
programming  (27).   Tested  effectiveness,  response  mode, 
prompts,  reinforcement  schedules,  etc. 

133.  Goldbeck,  R.A.  and  Cambell,  V.N. ,  "The  Effects  of  Response  Mode  and 

Response  Difficulty  on  Programmed  Learning"  Journal  of  Educational 
Psychology,  Vol.  53,  No,  3,  1962,  pps.  110-118. 

134.  Goldbeck,  R.A.  ,  et.al.,  "Integrating  Programmed  Instruction  with  Con- 

ventional Classroom  Teaching",  American  Institute  for  Research, 
AIR-CU9-12/62-FR,  December  1962. 

135.  Gulo,  E.V.  and  Nigro,  M.R, ,  "Classroom  Learning  as  a  Function  of 

Method  of  Presenting  Instructional  Materials",  Psychological 
Repts. .  Vol.  19,  No.  3,  Part  I,  December  1966,  pps,  971-977. 
Compared  the  teaching  efficiencies  of  PI,  television, 
and  conventional  texts.  -Conventional  text  group 
learned  the  most . 

136.  Herbert,  J,  and  Foshay,  A.W. ,  "Programmed  Instruction  in  the  Manhasset 

Junior  High  School",  from  Schramm  (l8l),  1964  9  pps,  18-27 > 

Discusses  the  administrative  problems  and  results  involved 
in  the  introduction  of  the  programmed  sequence  English  2600 
during  1960-63. 

137-   Herrick,  M.C.,  "The  Effect  of  Problem- Setting  Questions  en  Rate  and 
Amount  of  Learning  in  Programming  Teaching  Machines",  Office  of 
Education,  U.S.  Dept ,  of  H.E.W.,  Rept „  on  Grant  712130,  May  1962„ 

138.  Hershberger,  tf, ,  "Self-evaluational  Responding  and  Typographical  Cueing: 

Techniques  for  Programming  Self-Instructional  Reading  Materials", 
Journal  of  Educational  Psychology.  Vol.  55,  No,  5,  1964,  pps,  288-296 

139.  Hershberger,  W.A„  and  Terry,  D,F,,  "Delay  of  Self-Testing  in  Three 

Types  of  Programmed  Text",  Journal  of  Educational  Psychology, 
Vol.  56,  No.  1,  1965,  pps.  22-30* 

140.  Hessert,  R.B. ,  "A  Comparison  of  Three  Methods  cf  Programming  Subject 

Material  for  Auto-Instruction",  from  Glaser  fl32)„  196l,  pps,  97- 

114. 

Compared  a  linear  program  prepared  by  the  ruleg  method, 
a  prompting  form  of  this  program,  and  a  third  form  in  which 
multiple  choice  questions  were  used  which  had  to  be  answered 
correctly  before  continuing, 


lUl.   Holt,  H.O.,  "An  Exploratory  Study  of  the  Use  of  a  Self-instruction 

Program  in  Basic  Electricity"  from  Hughes  (U5),  1963,  pps .  15-58. 

1^2.   Hughes,  J.L.  ,  "Some  Aspects  of  IBM  Research  on  Programmed  Instruction" 
from  Hughes  (U5),  1963,  pps.  161-197. 

Describes  some  of  the  PI  techniques  used  at  IBM  for  employee 
training. 

lU3.   Hunt,  W.A.  and  Mathis ,,  C,  "The  Use  of  Programmed  Instruction  in 

Introductory  Psychology  for  Teachers",  Office  of  Ed.,  U.S.  Dept. 
of  H.E.W.,  Report  for  Nat.  Def.  Ed.  Act  of  1958,  Grant  No.  7-23- 
0790-207. 

ikk.      Huskey,  H.D. ,  "Automatic  Computers  and  Teaching  Machines",  from  Coulson 
(16),  1962,  pps.  257-272. 

Develops  a  compiler  for  computers  for  use  in  CAI. 

IU5.   Jeffels,  A.,  "Programmed  Techniques  and  Their  Use  in  Apprentice  Training" 
from  Unwin  (91),  1967,  pps.  135-1149. 

lh6.      Kazmier,  L.J.,  "Repeated  Exposure  to  Programmed  Instruction  and 

Student  Attitude",  Psychological  Reports,  Vol.  19,  No,  3,  Part  I, 
December  1966,  pps.  985-986. 

Repeated  use  increased  probability  of  a  favorable  attitude. 

1^7.  Keislar,  E.R. ,  "The  Development  of  Understanding  in  Arithmetic  by  a 
Teaching  Machine",  Journal  of  Educational  Psychology,  Vol.  50, 
No.  6,  1959,  pps.  2li7-253. 

lU8 .   Keislar,  E.R,  and  McNeil,  J.D.  ,  "A  Comparison  of  Two  Response  Modes  in 
an  Autoinstructional  Program  with  Children  in  the  Primary  Grades", 
Journal  of  Educational  Psychology.  Vol.  53,  No.  3,  1962,  pps.  127-131. 

1^9.   Krumboltz,  J.D.  and  Weisman,  R.G. ,  "The  Effect  of  Overt  vs.  Covert 
Responding  to  Programmed  Instruction  on  Immediate  and  Delayed 
Retention",  Journal  of  Educational  Psychology,  Vol.  53,  No.  2, 
1962,  pps.  89-92. 

150.  Krumboltz,  J.D.  and  Weisman,  R.G.,  "The  Effect  of  Intermittent 

Confirmation  in  Programmed  Instruction",  Journal  of  Educational 
Psychology,  Vol.  53,  No.  6,  1962,  pps.  250-253. 

151.  Leith,  G.O.M. ,  "The  Role  of  Overt  Responding  in  Programmed  Learning", 

from  Unwin  (91),  1967,  pps.  503-513. 

152.  Leonard,  J,M.  and  Wing,  R.L. ,  "Advantages  of  Using  a  Computer  in 

Some  Kinds  of  Educational  Games",  IEEE  Transactions  on  Human 
Factors  in  Electronics,  Vol,  HFE-8,  No,  2,  June  1967,  pps.  75-81. 
Believes  computer  is  better  than  teacher  in  economics 
oriented  educational  games.   Describes  two  implementations. 


153.   Levin,  G.R.  and  Bakers  B,L,,  "Item  Scrambling  in  a  Self-Instructional 
Program",  Brown  University  Report,  Providence,  R.I,,  1961,   Also 
in  Journal  of  Educational  Psychology,  Vol.  5^»  No.  3,  1963, 
pps.  138-1^3. 
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